Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs (Extended Version)

Many exact and approximate solution methods for Markov Decision Processes (MDPs) attempt to exploit structure in the problem and are based on factorization of the value function. Especially multiagent settings, however, are known to suffer from an exponential increase in value component sizes as interactions become denser, meaning that approximation architectures are restricted in the problem sizes and types they can handle. We present an approach to mitigate this limitation for certain types of multiagent systems, exploiting a property that can be thought of as "anonymous influence" in the factored MDP. Anonymous influence summarizes joint variable effects efficiently whenever the explicit representation of variable identity in the problem can be avoided. We show how representational benefits from anonymity translate into computational efficiencies, both for general variable elimination in a factor graph but in particular also for the approximate linear programming solution to factored MDPs. The latter allows to scale linear programming to factored MDPs that were previously unsolvable. Our results are shown for the control of a stochastic disease process over a densely connected graph with 50 nodes and 25 agents.Comment: Extended version of AAAI 2016 pape

Exploiting feature dynamics for active object recognition

This paper describes a new approach to object recognition for active vision systems that integrates information across multiple observations of an object. The approach exploits the order relationship between successive frames to derive a classifier based on the characteristic motion of local features across visual sweeps. This motion model reveals structural information about the object that can be exploited for recognition. The main contribution of this paper is a recognition system that extends invariant local features (shape contexts) into the time domain by integration of a motion model. Evaluations on one standardized and one custom collected dataset from the humanoid robot in our laboratory demonstrate that the motion model allows higher-quality hypotheses about object categories quicker than a baseline system that treats object views as unordered streams of images

Exploiting object dynamics for recognition and control

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. 127-132).This thesis explores how state-of-the-art object recognition methods can benefit from integrating information across multiple observations of an object. Considered are active vision systems that allow to steer the camera along predetermined trajectories, resulting in sweeps of ordered views of an object. For systems of this kind, a solution is presented that exploits the order relationship between successive frames to derive a classifier based on the characteristic motion of local features across the sweep. It is shown that this motion model reveals structural information about the object that can be exploited for recognition. The main contribution of this thesis is a recognition system that extends invariant local features (shape context) into the time domain by adding the mentioned feature motion model into a joint classifier. Second, an entropy-based view selection scheme is presented that allows the vision system to skip ahead to highly discriminative viewing positions. Using two datasets, one standard (ETH-80) and one collected from our robot head, both feature motion and active view selection extensions are shown to achieve a higher-quality hypothesis about the presented object quicker than a baseline system treating object views as an unordered stream of images.by Philipp Robbel.S.M

Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs

The Markov Decision Process (MDP) framework is a versatile method for addressing single and multiagent sequential decision making problems. Many exact and approximate solution methods attempt to exploit struc- ture in the problem and are based on value factoriza- tion. Especially multiagent settings (MAS), however, are known to suffer from an exponential increase in value component sizes as interactions become denser, meaning that approximation architectures are overly re- stricted in the problem sizes and types they can handle. We present an approach to mitigate this limitation for certain types of MASs, exploiting a property that can be thought of as ‘anonymous influence’ in the factored MDP. In particular, we show how anonymity can lead to representational and computational efficiencies, both for general variable elimination in a factor graph but also for the approximate linear programming solution to factored MDPs. The latter allows to scale linear pro- gramming to factored MDPs that were previously un- solvable. Our results are shown for a disease control do- main over a graph with 50 nodes that are each connected with up to 15 neighbors

Effective Approximations for Spatial Task Allocation Problems

Although multi-robot systems have received substantial research attention in recent years, multi-robot coordination still remains a difficult task. Especially, when dealing with spatially distributed tasks and many robots, central control quickly becomes infeasible due to the exponential explosion in the number of joint actions and states. We propose a general algorithm that allows for distributed control, that overcomes the exponential growth in the number of joint actions by aggregating the effect of other agents in the system into a probabilistic model, called subjective approximations, and then choosing the best response. We show for a multi-robot grid-world how the algorithm can be implemented in the well studied Multiagent Markov Decision Process framework, as a sub-class called spatial task allocation problems (SPATAPs). In this framework, we show how to tackle SPATAPs using online, distributed planning by combining subjective agent approximations with restriction of attention to current tasks in the world. An empirical evaluation shows that the combination of both strategies allows to scale to very large problems, while providing near-optimal solutions

Effective Approximations for Multi-Robot Coordination in Spatially Distributed Tasks

Although multi-robot systems have received substantial research attention in recent years, multi-robot coordination still remains a difficult task. Especially, when dealing with spatially distributed tasks and many robots, central control quickly becomes infeasible due to the exponential explosion in the number of joint actions and states. We propose a general algorithm that allows for distributed control, that overcomes the exponential growth in the number of joint actions by aggregating the effect of other agents in the system into a probabilistic model, called subjective approximations, and then choosing the best response. We show for a multi-robot grid-world how the algorithm can be implemented in the well studied Multiagent Markov Decision Process framework, as a sub-class called spatial task allocation problems (SPATAPs). In this framework, we show how to tackle SPATAPs using online, distributed planning by combining subjective agent approximations with restriction of attention to current tasks in the world. An empirical evaluation shows that the combination of both strategies allows to scale to very large problems, while providing near-optimal solutions

Local multiagent control in large factored planning Problems

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 137-145).Many problems of economic and societal interest in today's world involve tasks that are inherently distributed in nature. Whether it be the efficient control of robotic warehouses or delivery drones, distributed computing in the Internet of things, or battling a disease outbreak in a city, they all share a common setting where multiple agents collaborate to jointly solve a larger task. he ability to quickly and effective solutions in such multiagent systems (MASs) forms an important prerequisite for enabling applications that require flexibility to changes in tasks or availability of agents. his thesis contributes to the understanding and efficient exploitation of locality for the solution of general, cooperative multiagent Markov Decision Processes (MDPs). To achieve this, the proposed approximation architectures assume that the solution of the overall system can be represented with sparsely interacting (i.e., local) value function components that -- if found -- approximate the global solution well. Locality takes on multiple interpretations, from its spatial sense to more general sparse interactions between subsets of agents, and the efficient representation of local effects in large planning problems. Developed in the thesis are computational methods for extracting sparse agent coordination structure automatically in general, cooperative MDPs. Based on novel theoretical insights about factored value functions, the proposed algorithms automate the search for coordination via principled basis expansion in the approximate linear program (ALP). We show that the search maintains bounded solutions with respect to the optimal solution and that the bound improves monotonically. Introduced then are novel solution methods that exploit "anonymous influence" in a particular class of factored MDPs. We show how anonymity can lead to representational and computational efficiencies, both for general variable elimination in a factor graph but also for the ALP solution to factored MDPs. he latter allows to scale linear programming to MDPs that were previously unsolvable. Complex MAS applications require a principled trade-off between complexity in agent coordination and solution quality. he thesis results enable bounded approximate solutions to large multiagent control problems -- e.g., disease control with up to 50 agents in graphs with 100 nodes -- for which previously only empirical results were reported.by Philipp Robbel.Ph. D